Means and Ends: a Comparative Study of Empirical Methods for Investigating Governance and Performance

Scholars within different disciplines employ a wide range of empirical approaches to understanding how, why and with what consequences government is organized. We first review recent statistical modeling efforts in the areas of education, job-training, welfare reform and drug abuse treatment and assess recent advances in quantitative research designs. We then estimate governance models with two different data sets in the area of job training using three different statistical approaches: hierarchical linear models (HLM); ordinary least squares (OLS) regression models using individual level data; and OLS models using outcome measures aggregated at the site or administrator level. We show that HLM approaches are in general superior to OLS approaches in that they produce (1) a fuller and more precise understanding of complex, hierarchical relationships in government, (2) more information about the amount of variation explained by statistical models at different levels of analysis, and (3) increased generalizability of findings across different sites or organizations with varying characteristics. The notable inconsistencies in the estimated OLS regression coefficients are of particular interest to the study of governance, since these estimated relationships are nearly always the primary focus of public policy and public management research.